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Understand What Works in Postsecondary 
Education 


Authored by Alexander Mayer, MDRC, and Kate Tromble, Results for America 


A college degree remains critical to unlocking opportunity and to accessing 
America’s middle class, yet millions of students who pursue higher education 
never earn degrees. At community colleges, which disproportionately serve low- 
income students and students of color, only a third of students earn a two-year 
degree within three years. At four-year universities, only half of low-income 
students graduate within six years. The rising costs of college are one factor. 
These include not only tuition, but also transportation, textbooks, housing, and 
food, which together often exceed tuition. Academics are another factor. Two- 
thirds of entering freshmen in community colleges are identified as 
academically underprepared and are required to take remedial courses that do 
not count toward a degree. Most never graduate. These students face other 
barriers too, including balancing work and family, lack of information, lack of 
social and family resources to navigate college, and institutional policies and 
practices that create obstacles—all of which have been exacerbated by 
COVID-19. 


This memo draws lessons and policy implications from two decades of rigorous 
research in postsecondary institutions focused on these problems. The students 
who participated in this research were primarily low-income students and many 
were students of color. All of the studies referenced here used randomized 
controlled trials, the highest standard of evidence recognized by the U.S 
Department of Education and the scientific research community. Despite the 
barriers that many students face, the findings discussed here demonstrate the 
growing and impressive evidence base on what can be done to help low-income 
students and students of color succeed in college. 


Key Lessons for Higher Education 


Lesson 1: Frequent, proactive advising tied to financial support is 
highly effective, improving credit accumulation and persistence, 
and in some cases driving large improvements in graduation. 


By themselves, short-term interventions like financial awards and enhanced 
advising have small effects. When these strategies are combined in more 
comprehensive programs, however, they can drive large improvements in 
graduation. The City University of New York’s (CUNY) Accelerated Study in 
Associate Programs (ASAP) is among the most effective and comprehensive 
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programs that have been studied. CUNY ASAP is a three-year program. It 
requires students to enroll full time and frequently meet with advisors who have 
small caseloads. Students must also participate in tutoring, enhanced career 
services, and block-scheduled courses for the first year. They receive a tuition 
waiver to fill any gap between financial aid and tuition and fees, free MetroCards 
for public transportation, and free use of textbooks. ASAP nearly doubled 
graduation rates in an independent evaluation at CUNY, with consistent results 
across racial groups. ASAP also doubled graduation rates again when it was 
replicated and tested in three Ohio colleges. 


Other studies reinforce the importance of multi-year programs that combine 
advising with additional financial aid. Short-term advising and coaching 
programs can help students make academic progress (e.g., Inside Track; or 
Opening Doors, which also included small stipends) but do not always produce 
measurable gains (e.g., Monitoring Advising Analytics to Promote Success 
and iPASS). Additional financial aid can also improve outcomes, but impacts are 
smaller than combined programs (see below). Evidence about comprehensive 
programs that combine frequent advising and additional financial aid, however, 
is accumulating, especially for studies at early stages and for short-term 
outcomes. These include rigorous studies on the Valley Initiative for 
Development and Advancement, Bottom Line, One Million Degrees, and 
the Detroit Promise Path, which all show positive short term impacts. Positive 
findings are also being seen in rigorous studies of programs that provide 
counseling or advising and additional financial support, with a focus on certain 
employment sectors, such as Project Quest, Year Up, I-BEST, Health Careers 
for All, and Bridge to Employment in Healthcare. 


Implementing comprehensive, multi-year advising programs, however, can be 
costly. Most of the evidence cited above comes from programs operating at 
community colleges, which tend to be resource-constrained and are likely to 
remain so throughout the COVID-19 pandemic and its aftermath. 


Lesson 2: Additional financial aid can help low-income students 
succeed, but decisions about how the program is designed are 
important. 


Increased financial aid (on top of Pell Grants and other aid) in the form of 
scholarships and grants is an important component of the most successful 
strategies that have been tested so far. Although many studies attempt to 
determine the causal effects of grant aid independent of other factors, far fewer 
use randomized controlled trials (RCT) to provide the most reliable 
estimates. The strongest research available in the U.S. includes large evaluations 
in nine states: privately-funded grants to Nebraska high school students to 
attend in-state public postsecondary institutions; privately-funded grants to Pell- 
eligible Wisconsin residents who were already enrolled at four-year public 
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institutions; performance-based scholarships for low-income parents attending 
community colleges in Louisiana, in which a portion of each grant was 
conditioned on the student’s performance in the semester; and a large-scale 
demonstration project to replicate and adapt the Louisiana performance-based 
scholarships with a diverse range of low-income students in Arizona, 
California, Florida, New Mexico, New York, and Ohio. These studies 
demonstrate that grants of different designs and amounts can improve college 
access and progress towards graduation, and in some cases they can modestly 
improve completion. 


Increasing financial aid, however, does not always work. The Wisconsin program 
described above included a similar scholarship for students at Wisconsin 
community colleges, also tested in an RCT. The community college scholarship 
did not measurably improve student outcomes, a result that the authors 
attribute partially to small complexities in implementation that created barriers 
for students, and partially to negative effects the offer appeared to generate for a 
subsample of students who turned out to be ineligible for the award. Likewise, 
the promise of a scholarship worth up to $12,000 to students who graduate 
from Milwaukee public high schools has not produced demonstrably positive 
impacts on college enrollment, potentially related to program design decisions 
like high school performance requirements and insufficient attention to school 
culture and resources to catalyze student interest. When they are well-designed 
and meet the needs of the target student population, scholarships and grants can 
be important strategies to help students succeed. 


Lesson 3: Developmental or remedial education reforms can help 
more students succeed in college-level courses. 


Millions of college students are required to pass developmental or remedial 
classes before they can enroll in credit-bearing courses at the college level, but 
most never graduate. Rigorous research studies conducted at seven colleges in 
New York and five colleges in Minnesota and Wisconsin, however, show 
that simply using better data to determine college readiness, such as students’ 
high school GPA, can help many students skip remedial courses altogether and 
still succeed in college-level courses. Intensive instruction and support programs 
can also help students with multiple remedial needs in reading, writing and 
math. CUNY Start, for example, substantially increases the proportion of 
students who complete developmental education. CUNY Start condenses the 
time students spend preparing for college English and math into one semester, 
before formal matriculation; integrates content areas like reading and writing 
using engaging instructional practices; includes student supports like tutoring 
and skills seminars during classroom time; and provides intensive professional 
development for faculty. 
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Other instructional reforms are improving outcomes as well. The Dana Center’s 
Mathematics Pathways (DCMP) improved the proportion of students who 
passed developmental and college-level math courses in a study at four 
community colleges in Texas. DCMP is a major instructional reform of 
developmental or remedial math. Instead of teaching algebra to all students, the 
program offers statistics and quantitative pathways that align with students’ 
majors. Instructors also avoid lecturing and focus on more interactive forms of 
student learning. 


Evidence is also growing around corequisite remediation, in which students 
enroll directly in college-level courses, in addition to courses that provide 
concurrent support for developmental or remedial needs. A rigorous study of 
corequisite remediation in English at five Texas community colleges found large 
gains in the percentage of students who passed college-level English, with 
consistently positive impacts for traditionally underrepresented groups. In math, 
CUNY created a program that combined features of math pathways and 
corequisite remediation: instead of developmental algebra, students took 
college-level statistics with corequisite remediation. A rigorous study found that 
the program not only improved college-level math completion, but also 
improved graduation. Measurable increases in graduation, however, are rare 
for developmental or remedial education reforms. In general, the reforms 
discussed here can help more students pass college-level courses and advance 
towards graduation, but most likely need to be paired with other strategies to 
substantially improve completion. 


Lesson 4: Behavioral strategies can increase participation in 
activities that promote student success. 


Insights from behavioral science can help identify and develop strategies to 
overcome barriers that may prevent student success. One prominent study 
demonstrated that simply providing people with personalized help to fill out the 
federal Free Application for Financial Student Aid (FAFSA) increased the 
percentage of low-income families who submitted the FAFSA, increased the 
amount of financial aid their students received, and impacted college access and 
persistence. Strategies that include simple adjustments to present information 
more clearly or more saliently—or simple reminders to complete required 
documents—can also boost the number of students who engage in activities 
associated with college completion. In one study, students who received emails 
and mailings about summer enrollment took summer classes at higher rates and 
earned more credits, with results that were even larger when paired with a 
modest amount of additional financial aid. Research has also found that simply 
changing default loan offers from $0 to any nonzero amount, or providing 
information about federal student loans, can change student behavior and 
increase borrowing. 
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On the other hand, behavioral messages will not achieve their desired effect if 
they are not appropriately tailored to the target population or do not resonate 
with students. Two large-scale studies of messages that were designed to help 
students apply for financial aid, for example, did not increase FAFSA submission 
rates. Similarly, although an initial study found that customized information 
can help high-achieving, low-income students apply to and enroll in 
academically stronger institutions, large scale follow-up research did not 
confirm these results. When behavioral messages are well-designed for the 
target population, however, they can be compelling approaches for colleges to 
adopt or pair with other strategies. Moreover, they usually cost very little, so even 
small effects on enrollment can produce increases in tuition revenue that more 
than cover the cost of the intervention. 


Policy Implications 


1. Combine proven strategies like frequent, proactive advising and 
additional financial support. Programs that, in isolation, have small 
short-term effects can still dramatically improve outcomes when 
combined with other effective programs. Frequent, proactive advising 
combined with financial support are central to the most effective 
programs that have been rigorously evaluated. Programs sometimes 
include other supports like tutoring or career counseling. Sustaining these 
programs over multiple years is also a key factor contributing to their 
success, and behavioral strategies can help ensure students participate in 
important activities. For policymakers, this means encouraging and 
incentivizing institutions to implement a comprehensive, evidence-based 
approach to supporting students. 


2. Prioritize equity in completion efforts. Low graduation rates are a 
problem of national urgency, concentrated predominantly among low- 
income students and students of color. The programs included in this 
memo were all evaluated in independent, rigorous evaluations and tended 
to work for diverse student populations. Giving substantially more 
students of color and low-income students access to the most effective 
programs could dramatically reduce equity gaps in the U.S. 


3. Provide additional federal resources to finance proven strategies. 
Implementing proven programs costs money. Community colleges, in 
particular, tend to be resource-constrained and are likely to remain so 
throughout the COVID-19 pandemic and its aftermath. Few have budgets 
for resource-intensive programs, even those that have been shown to 
positively impact student success. Solving the college completion problem 
will require additional governmental support tied to rigorously tested 
interventions that are demonstrated to improve outcomes for low-income 
students and students of color. 
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Design a Rigorous Postsecondary Learning 
Agenda 


Authored by Clare McCann, New America 


A well-designed learning agenda can help to drive policy throughout an agency, 
promote the development and use of evidence by agency officials, and ensure the 
research and policy arms of the agency are rowing in the same direction. The 
Education Department’s learning agenda should reflect Department priorities, 
identify major gaps in existing research that can be filled by the Department, and 
inform policy iteration within the agency. 


Learning agendas are also required under the Foundations for Evidence-Based 
Policymaking Act of 2018 (the Evidence Act), legislation designed to act on some 
of the foundational recommendations of the Commission on Evidence-Based 
Policymaking and led in the Senate by Senate Health, Education, Labor, and 
Pensions (HELP) Committee Ranking Member Patty Murray. Along with the 
assignment of key personnel to implement evidence-driven policies (e.g., a Chief 
Data Officer) and the creation of evaluation and open data plans, it is the first 
stage of implementation of the Evidence Act. Agencies are already underway in 
designing their learning agendas. Higher education must be a key part of the 
Education Department’s agenda. 


Postsecondary education is facing critical problems. The postsecondary learning 
agenda proposed below seeks to identify key areas where new research could 
help drive important improvements and inform the basis of future policymaking. 
It highlights existing deficiencies and gaps in available data, insufficient research 
and evidence about how best to help students, and unanswered questions that— 
with more and better research behind them—could drive greater student success. 
In particular, it seeks to identify where additional evidence can inform an agenda 
of an accessible, affordable, and equitable postsecondary education that places a 
premium on student success. 


Postsecondary Learning Agenda 


This proposed learning agenda addresses some of the biggest challenges and 
problems the U.S. faces in postsecondary education: access to higher education is 
inequitable; college is increasingly unaffordable, particularly for low-income 
students; students (particularly students of color) often struggle to succeed in 
higher education; and institutions too often fail to meet their students’ needs or 
leave students without credentials of value. Already, we know that students of 
color are drastically less likely to go to college at all, or to go to a college that gives 
them a good chance of success. Low-income students face significant challenges 
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—financial and otherwise—in completing college, or completing on time. Black 
students are particularly burdened by student loan debt, as are students who 
leave school with debt and no degree. Outcomes and return on investment vary 
drastically across institutions and programs, and across student subgroups within 
those programs. 


Yet past attempts to solve these problems, including through grants that seed 
innovation and accountability measures that raise expectations for institutions, 
have often been disjointed and short-lived, falling short of what is needed to 
ameliorate these urgent national problems. A well-designed learning agenda can 
help identify workable solutions, addressing different perspectives when 
answering questions, and inform how best to use resources. 


To begin developing the postsecondary agenda, first and foremost, the 
Department should seek to tap the expertise of researchers in the evidence and 
higher education communities, as well as determine the types of evidence that 
the Department will seek to build (including tapping the most-rigorous 
randomized control trials, but also allowing for different types of research to 
contribute to answering these questions). The Department should marshal the 
evidence that is already available with respect to each of these research areas and 
assess where the research is deficient or insufficiently nuanced to inform policy 
and practice. Outreach to engaged stakeholders like philanthropic foundations 
that are invested in building evidence and improving outcomes can help to invite 
informed input, align the field around these critical issues, and marshal the 
resources necessary to bring it to fruition, even in the absence of sufficient 
federal resources. Finally, the Department should identify a strategy for investing 
in evidence-building for each of these areas through its own work, both at IES 
and within the agency more broadly. 


This learning agenda is divided into nine separate research questions across 
those four categories: access, affordability, student success, and institutional 
outcomes. The research questions are designed to answer critical questions about 
postsecondary success, including through access to data held by the U.S. 
Department of Education that have not been published or extensively mined for 
insights, as well as by inviting creative thinking about outside sources of data that 
can help to fill other gaps in existing research, with a particular emphasis on 
equity. Research areas like these will help the Department to understand the 
extent and nature of the problem, identify the evidence that exists and that is 
missing, identify opportunities for improvement, establish best practices for 
institutions seeking to improve their students’ odds of success, drive future policy 
change, and improve the management of taxpayer-funded student financial aid 
programs. 


- Access to higher education is inequitable. Access to higher education 
has improved significantly over the decades, with nearly 70 percent of 
recent high school graduates now enrolling in college (compared with 45 
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percent in 1960), and millions of working adults returning to 
postsecondary education. Yet that access is far from evenly spread. 
Students of color are less likely to attend institutions and enroll in 
programs that offer them the greatest returns. For instance, while 35 
percent of white adults had completed a bachelor’s degree in 2016, far 
fewer Black (21 percent) and Hispanic (15 percent) adults had. 


o What does the evidence indicate are the characteristics of effective 
college-access programs, particularly for people of color and low- 
income people? 


- College is increasingly unaffordable, particularly for low-income 
students. In the decades since the federal government began awarding 
aid to help students attend college, the average tuition and fees of 
institutions have increased substantially, growing at well beyond the rate 
of inflation, while federal student aid has not kept pace. At the same 
time, many states have failed to invest adequately in public higher 
education; costs have been shifted onto students and their families; and 
much of the gap in costs has been borne through student debt. 


° How do student financial grants, particularly federal need-based 
grants—including the knowledge of those grants’ existence—affect 
indebtedness, access, attendance patterns (such as sector, 
intensity, and stop-out periods), retention, and completion? 


° To what extent are student loan impacts on access, retention, 
completion, repayment, and household financial stability different 
across student financial circumstances, race/ethnicity, gender, and 
repayment plans? 


fo} 


What strategies are effective in helping students to avoid 
unaffordable debt and avoid default on their loans, and to resolve 
delinquencies and defaults quickly with minimal negative impact 
on borrowers’ finances? 


° To what extent do long-standing disparities in family wealth and 
income relate to college debt, including intergenerational debt and 
private education loans? 


- Students (particularly students of color) often struggle to succeed 
in higher education. Millions of students who pursue higher education 
never pursue a degree; many leave school with debt anyway. At 
community colleges, which disproportionately serve low-income students 
and students of color, only a third of students earn a two-year degree 
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within three years. At four-year universities, only half of low-income 
students graduate within six years. While there is some evidence about 
the types of programs that are most effective in helping students with 
financial issues, academic challenges (like developmental education 
programs), and other non-academic supports (like advising), greater 
research is needed into how to knit effective reforms together into a 
holistic approach to supporting students and expanding those reforms to 
more institutions and student populations. 


o What strategies (including non-financial interventions) does the 
evidence indicate are effective, and in what contexts, in supporting 
college completion, particularly for low-income students, students 
of color, and academically underprepared students? 


fo} 


What are distinguishing characteristics of institutions and 
programs that have successfully promoted improvement in student 
outcomes, particularly in closing gaps in performance across 
student subgroups? 


° To what extent are different types of credentials and programs, 
including similar programs across different institutions, associated 
with positive labor market outcomes, including employment, 
earnings, and improvements in institutional earnings? 


- Institutions too often fail to meet their students’ needs or leave 
students without credentials of value. Taxpayers fund more than 
5,000 institutions of higher education across the nation, spanning a 
substantial diversity of size, program offerings, effectiveness, modality, 
and more. Yet many of those colleges leave students with poor outcomes, 
where a significant percentage of students drop out and/or wind up in low- 
wage jobs, no better off than if they had not enrolled in college in the first 
place. Some colleges wind up wasting taxpayer dollars; and in recent 
years, hundreds of campuses have closed down, often without notice to 
students. Department policies provide too little incentive for institutions 
to meet a minimum bar of quality or to improve. 


c How effectively do various metrics of financial viability, student 
outcomes, and other elements of risk identify institutions at risk of 
closure, low-value education, and/or other liabilities for taxpayers? 
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Establish a Tiered Evidence Fund for Innovation 
and Expansion 


Authored by Roger Low and Karen Quarles, America Forward, and Kate Tromble, 
Results for America 


Today’s postsecondary students are as varied as our nation, encompassing 
individuals just out of high school and older, working (full or part time) adults. 
Thirty-one percent are at or below the poverty line, only 46 percent live on 
campus, and nearly 30 percent have children of their own. Furthermore, these 
students are increasingly likely to come from Black and brown communities, 
and often face systemic barriers that make it harder to complete their 
postsecondary degrees, compared with many of their peers. 


Each year 1.2 million low-income or first-generation students enroll in college, 
but only a quarter of them will leave with a job related to their studies or enter 
graduate school. Children who have never lived in poverty are ten times more 
likely to complete a college degree by age 25, compared with children who grow 
up persistently poor. Even at the most affordable community colleges, only 
about one-third of students graduate within one-and-a-half times the length of 
the program. 


Well before the COVID-19 crisis brought about new challenges for students, it 
was clear that empowering students to succeed in postsecondary education and 
beyond was critical to increasing equitable economic mobility. With the spread of 
COVID-19, and the disparate impacts it has wrought on Black, brown and low- 
income communities across the country, the need for reform is more urgent than 
ever. 


If we want to make higher education truly an engine of opportunity for low- 


income students and students of color’ we need to ensure that colleges across the 
country are adopting evidence-based strategies to improve results. This requires 
multiple strategies. We need institutions to come up with new interventions and 
supports, and evaluate them to see if they work. Ideas with some evidentiary base 
should be evaluated in multiple contexts and settings to ensure they work in 
different environments. And those ideas that have been proven time and again to 
work need a path to adoption on a national scale. 


The Department of Education can play a key role in facilitating the development 
and spread of innovative practices by adopting a tiered grant competition. In 
2009, the American Recovery and Reinvestment Act (ARRA) was enacted to 
address the economic fallout from the Great Recession. The law went further 
though, encouraging and supporting innovation in education through the 
evidence-based Investing in Innovation (I3) Fund and its successor, the 
Education, Innovation and Research (EIR) program at the same time it 
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addressed immediate economic needs.” Several years later, the First in the World 
(FITW) program was created with the goal of developing, replicating, and 
disseminating innovative solutions and evidence for what works in addressing 
persistent and widespread challenges in postsecondary education for vulnerable 
students. Given the burden the current pandemic is placing on our Black, brown 
and low-income communities in particular, the next Administration should build 
on these programs by prioritizing the proposed college completion grants, based 
on levels of evidence, in its first budget proposal. 


Invest In and Scale Up Effective Approaches 


In recent years, a handful of interventions or approaches in the higher education 
domain have been rigorously evaluated, replicated, and shown to be effective at 
measurably improving student postsecondary degree attainment, particularly for 
low-income students and students of color. For instance: 


- City University of New York (CUNY) Accelerated Study in Associate 
Programs (ASAP): The intervention is “a comprehensive set of 
supports that include financial assistance, advising and career counseling, 
tutoring, training in study skills, and reformed remedial courses,” aimed 
at first-time students, students of color, students in or near poverty, and 
students with high risk factors. A rigorous 2015 MDRC evaluation found 
that within three years, ASAP nearly doubled graduation rates, from 22 
percent to 40 percent. The program was expanded to community colleges 
in Ohio; a second randomized evaluation from Ohio, equally rigorous, 
released in early 2020, shows equally positive results, with robust and 
comparable effects across white students and students of color. 


Bottom Line: Like CUNY ASAP, Bottom Line provides intensive, 
individualized guidance and support for low-income, first-generation 
students. Results for America found that a well-conducted 
randomized trial showed that students offered Bottom Line advising 
were substantially more likely to be enrolled in college at the three-year 
follow up with robust and comparable effects across white students and 
students of color. 


Project Quest: QUEST provides comprehensive supports for low- 
income individuals to enroll “full-time in occupational training programs 
at local community colleges, complete the training, pass certification 
exams, and enter well-paying careers in high-growth sectors of the local 
economy” (such as for instance the healthcare sector). This program also 
requires full-time enrollment, and includes “weekly group or individual 
counseling sessions, financial assistance for tuition and other school- 
related expenses, and remedial instruction for those requiring it.” A 
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rigorous randomized trial found Quest “increased average annual 
earnings by $5,490, or 20 percent, nine years after program entry.” 


I-Best and Other Classroom-Based Interventions: The Washington State 
Integrated Basic Education and Skills Training (I-BEST) program “aims to 
increase access to and completion of college-level occupational training in 
a variety of in-demand occupational areas. Its signature feature is team 
teaching by a basic skills instructor and an occupational instructor during 
at least 50 percent of occupational training class time.” A rigorous 
randomized evaluation found that the I-BEST programs at the three 
technical and community colleges “increased participation in college level 
courses, number of credits earned and credential attainment.” In 
addition, certain other remedial education programs, such as CUNY 
Start’s remedial program, successfully increased student passage of 
gateway English and math courses in their first two years, according toa 
study with a matched comparison group. 


Additional Promising Programs: These include the Valley Initiative for 
Development and Advancement (VIDA), which combines career 
counseling, case management, and soft skills training; Detroit Promise 
Path, which combines a last dollar scholarship and advising; and Stay the 
Course, which combines comprehensive case management with 
emergency financial assistance. These programs take different but aligned 
approaches to ASAP. Notably, they also require full-time enrollment. 
VIDA and DPP have shown preliminary but promising results. 


Other Evidence-Building And Evidence-Scaling Efforts: A number of other 
interventions and organizations offer a related comprehensive set of holistic 
supports for higher education students, including Braven, College Forward, 
One Million Degrees, uAspire, Year Up, Health Careers for All, Bridge to 
Employment in Healthcare, and Single Stop. These programs are in varying 
stages of the evidence-building continuum from some promising programs that 
have not yet been rigorously evaluated, to strong early evidence (for instance, 
One Million Degrees) to established strong evidence (for instance, Year Up). 


There are no guarantees that interventions will succeed when it comes to 
something as complex as changing the trajectory of a student’s life. Different 
communities present different challenges. Demonstrating outcomes over one or 
even multiple evaluations is no guarantee of future success. Implementation 
matters. Community buy-in matters. Leadership and personnel matter. Any 
number of variables can complicate the scaling of even the most rigorously 
evaluated approaches — inadequate funds, fragmentation of the social 
innovation “ecosystem,” or gaps in the talent needed to scale, to name just a few. 
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With all of these caveats, the approaches summarized above, while intensive and 
expensive on the front end (ranging from $700 to $16,000 per participant per 
year), have demonstrated (or are in the process of seeking to demonstrate) a 
substantial return on investment (ROI) to the public sector and society in the 
long run. They offer hope that we can empower many more individuals to 
graduate from college and achieve greater economic security. And yet, although a 
handful of public and community colleges have funded these interventions, with 
impressive results, the vast majority of jurisdictions are not deploying public 
dollars to expand the supports identified by rigorous research as most effective at 
helping students with the highest barriers to successfully reach the collegiate 
finish line. 


In a 2019 report, New America observed: “the research on what works to 
improve students’ odds of success in higher education remains limited. Only a 
handful of organizations and researchers even conduct such evaluations... and 
there has been a lack of coordination by the federal government to solve the 
research problems or tackle the challenges of increasing college completion rates 
and other problems. Moreover, even when promising interventions have been 
identified, colleges have few incentives to apply that research. Many of these 
interventions require significant upfront costs, and the benefits and savings to 
campuses, which are often realized through increased retention and graduation 
rates, may take a long time to achieve. Federal policy requires only that colleges 
avoid extremely high default rates and comply with federal rules and regulations, 
few of which are based on outcomes. Federal dollars to support improvement and 
quality assurance are limited... As a result, practitioners often lack the resources 
—or incentive—they need to implement such practices.” 


Finally, as noted, today’s students are not the “traditional” college students that 
label calls to mind. They hold jobs, live off campus, face acute economic, 
housing, and transportation challenges, and have young children of their own. 
These students are also disproportionately not white, and they urgently need 
additional support. Yet, we have remarkably little rigorous evidence on 
approaches that successfully move the needle on degree attainment outcomes for 
part-time students. 


Tiered Innovation Fund 


- Arevised and expanded version of the FINISH Act fund located in 
an Office of Evidence-Based Practices, a competitive evidence-based 
innovation and replication grant program focused on increasing 
completion rates. Based at the U.S. Department of Education, the fund 
should start with an $800 million federal investment and be overseen by 
an Office of Evidence-Based Practices. The Department currently 
maintains an evidence-based practices team within the Office of 
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Elementary and Secondary Education. That team should be converted 
into an office serving both K-12 and postsecondary education and 
reporting directly to the Deputy Secretary. 


- Tap the right talent: In addition, we urge the office to assemble a mix of 
permanent experienced full-time staff, supplemented with targeted 
outside experts in the evaluation, evidence, higher education and social 
innovation fields. The Department could draw this talent into the newly 
constituted office via a combination of direct hire authorities and 
extended fellowships, including Intergovernmental Personnel Act (IPA) 
partnerships, leveraging authorities available to the Secretary under 


current law (additional information on these authorities is available here). 


- A well-structured fund: Theoretically, this fund could be placed within 
the Fund for the Improvement of Postsecondary Education (FIPSE), 
following the model of the FITW program. Our concern with adding 
dollars to FIPSE or the pre-existing FITW fund is three-fold: 


o First, those programs lack the intensive focus on college 
completion that we envision. 


fe} 


Second, they funded one-off, individual projects at institutions and 
nonprofit organizations, creating a myriad of pilots. More impact 
can be achieved by funding states working across their institutions 
in coordination and collaboration with nonprofit partners. 


oO 


Third, and perhaps most importantly, in some respects, FITW may 
have been ahead of its time. At the time FITW grants were 
awarded, there simply were not many interventions in the higher 
education space that had convincingly reached a top evidence tier. 
While FITW helped spur innovation, in hindsight, its grant awards 
arguably were inadequate and insufficiently targeted toward 
moving the most promising evidence-based programs further 
along an evidence continuum. Since the FITW grant awards, a 
range of compelling higher education interventions summarized 
above have made exciting strides, with the release of a new tranche 
of randomized and quasi-experimental studies. We believe the 
field has advanced rapidly in the last six years, and the time is now 
ripe for a major federal investment in expanding these programs, 
and building evidence for a new cohort of innovative approaches, 
leveraging the important lessons learned. In short, we think 
starting fresh, given how much the field has advanced and how 
much we have learned from evidence-based programs over the last 
decade, will offer a better chance for success. 
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4 Strong Evidence: 
Programs shown in well- 
conducted randomized 
controlled trials (RCTs), 
carried out in typical 
community settings, to 
produce sizable, sustained 
effects on important 
outcomes. This category 
requires replication— 
specifically, the 
demonstration of such 
effects in two or more RCTs 
conducted in different 
implementation sites, or, 
alternatively, in one large 
multi-site RCT. 


5 Preliminary Evidence: 
The model has evidence 
based on a reasonable 
hypothesis and supported 
by credible research 
findings. Such evidence 
suggests the program may 
be an especially strong 
candidate for further 
research but does not yet 
provide confidence that the 
program would produce 
important effects if 
implemented in new 
settings. 


6 Promising Evidence: 
Interventions that have 
been evaluated in RCTs or 
rigorous quasi-experimental 
studies, and found to have 
positive effects that are 
sizable, but not yet 
conclusive (e.g., due to only 
short-term follow-up, a 
single-site study design, 
well-matched comparison 
groups but not 
randomization, or effects 
that fall short of statistical 
significance). 
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- Adequate funding: FITW received less than $150 million over its two 
years of funding. The Social Innovation Fund (SIF), which helped launch 
compelling community-based solutions in over 100 cities in 33 states and 
the District of Columbia, received approximately $400 million in federal 
investment. When combined with philanthropic funds and other dollars, 
SIF reached an impact of over $1 billion. We believe SIF’s total funding 
level offers more potential for success. And, although we think that level 
can eventually be achieved through a matching requirement, we believe 
additional investment is needed initially to achieve sufficient scale given 
the financial strain many states and institutions are experiencing currently 
and the scale of the problem the fund seeks to address. Grants would be 
awarded to states or state-entities in partnership with their institutions 
and nonprofits operating evidence-based programs, interventions or 
policies that increase completion rates for at-risk students such as 
counseling, mentoring, financial assistance, and other supports. As a 
tiered evidence fund, grants would be awarded in ranges based upon the 
amount and quality of causal evidence underlying the state’s proposed 
plan. Grants at the highest, or “strong evidence,”* tier would be reserved 
for states proposing to scale an existing program that has met the What 
Works Clearinghouse’s (WWC) highest level of effectiveness rating. 
Programs would be required to submit their randomized or quasi- 
experimental evaluation evidence to the Institute of Education Sciences 
(IES), and for IES to review those evaluations within the first, planning 
year. Grants at the lowest, or “preliminary evidence,” tier would be for 
states proposing a program and strategic plan that would build evidence 
around how to increase completion for at-risk students. These grants 
would have to be accompanied by a plan for evaluating, in a rigorous 
manner, the impact of the proposed program, practice intervention or 
policy. The middle, or “promising evidence,”® tier would be for states 
proposing a plan to replicate an existing strategy that has shown positive 
causal impacts, but only has short-term, non-experimental results, or has 
not yet been replicated. 


- Planning Grants. Planning grants would be awarded in the range of 
$300,000 for a 12-month period to any state or state-entity interested in 
applying for a grant from the fund. Following a streamlined, rapid review 
and certification process by the U.S. Department of Education, any state 
or state-entity interested in submitting a competitive application would be 
eligible to automatically receive a planning grant, ensuring every state 
would have the resources, staff and capacity necessary to formulate a 
catalytic plan to build evidence for and scale effective higher education 
approaches. Recipients would develop a plan for either scaling an existing, 
WWC-reviewed, evidence-based approach or for building evidence of a 
new approach to increasing college completion; States could also elect to 
pursue a hybrid approach with a plan that did some of both. Plans would 
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need to identify, among other things, the state’s plan for distributing funds 
among its institutions to focus on the most at-risk students, its evidence- 
based intervention or program, its technical assistance/ implementation 
partner, its goals for increased completion, and its plan for tracking and 
publicly reporting its performance against those goals. 


Key Elements to Include 


- Focus on those most in need. State plans should specifically focus - as 
does the current FINISH Act - on first-generation students, students of 
color, and students in or near poverty, as well as students with high risk 
factors for not finishing college, such as those with children of their own 
and non-dependent, SNAP eligible students. To that end, state plans 
should prioritize partnerships with institutions that serve these students, 
namely community colleges, regional public universities, and minority- 
serving institutions. States that successfully prioritize these students, 
communities and institutions should receive favorable consideration in 
the competitive application process. Moreover, for grants at the early- 
phase or evidence-building tier, priority should be given to proposals to 
build evidence around increasing completion rates for part-time students 
with other vulnerability factors. 


Matching funds: Grantees also would need to provide matching dollars, 
which could be a combination of state, institution, and philanthropic 
dollars. States should be eligible for a waiver from the match requirement, 
through 2022 due to the current unfolding recession, with modifications as 
needed depending on the fiscal realities states face in 2022 and beyond. 


Funding for evaluation, performance management and capacity 
building: Costs associated with evidence-building and high-fidelity 
implementation of evidence-based programs (such as data system 
improvements, performance management improvements, technical 
assistance, evaluations and other training and capacity-building efforts) 
should be allowable uses of up to 10 percent of the full grant. (In the case 
of the planning grant, states should have maximal flexibility to use grant 
dollars for staff, planning, and capacity-building as needed to develop a 
coherent plan and application.) 


Deadlines for releasing guidance and selecting awards: So that the 
U.S. Department of Education prioritizes this initiative, legislation should 
include deadlines for the Department to release guidance, a Notice of 
Funding for the competitive grant process, and select awards. If the 
Department misses any of these deadlines by more than 60 days, the 
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legislation should require an investigation by the Office of the Inspector 
General into the cause of the delay. 


Prioritize an expanded FINISH Act in 117th Congress 


In addition, the new Administration should work closely with the bipartisan 
FINISH Act sponsors, as well as additional, engaged members of both parties (in 
particular, members of the relevant education and appropriations Committees in 
the House and Senate) to introduce an expanded and retooled FINISH Act in 
2021 that creates the evidence-based college completion fund proposed above. 


In 2019, Senators Todd Young (R-IN), Michael Bennet (D-CO), Tim Scott (R-SC) 
in the Senate and Representatives Ben McAdams (D-UT) and Ron Wright (R- 
TX) in the House introduced the Fund for Innovation and Success in Higher 
Education Act (FINISH Act), which included a new tiered-evidence fund to 
improve graduation rates and boost the attainment of postsecondary credentials 
for today’s students. These members can be tapped as possible champions for the 
legislation. 


The FINISH Act would amend the Fund for the Improvement of Secondary 
Education (FIPSE) within the Higher Education Act (HEA) to “allow grant 
recipients to use a portion of their funds for tiered, evidence-based grants to 
support initiatives that increase access to higher education for high-need 
students, increase degree attainment, and improve efficiency in our higher 
education systems.” 
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Notes 


1 One study, https://www.brookings.edu/research/ 
community-college-completion-rates-structural-and- 
motivational-barriers/#footref-7 found that 
completing a two-year associate’s degree is 
associated with a 37 percent increase in earnings 
compared with a high school diploma. Another 
recent analysis, https://research.collegeboard.org/ 
pdf/education-pays-2019-full-report.pdf#page=4 
found median earnings of bachelor’s degree 
recipients were $24,900 higher than those of high 
school graduates. Without a college degree, children 
born into a low-income family have a 45 percent, 
https://www.brookings.edu/wp-content/uploads/ 
2016/06/THP_13EconFacts_FINAL.pdf#page=16 
chance of remaining in the bottom quintile of 
earnings as an adult. With a college degree, those 
odds plummet to 16 percent., https:// 
www.brookings.edu/wp-content/uploads/2016/06/ 
THP_13EconFacts_FINAL.pdf#page=16 


2 Asimilar program targeted at identifying, 
promoting and scaling evidence-based programs 
across the social innovation sector, the Social 
Innovation Fund (SIF) was created in the same year. 
SIF and i3 funded innovative new projects coupled 
with rigorous evaluations at a “lower” evidence tier, 
while scaling proven approaches in “higher” evidence 
tiers. SIF also combined private dollars with public 
ones to extend the reach of evidence-based federal 
funds., https://obamawhitehouse.archives.gov/ 
administration/eop/sicp/initiatives/social-innovation- 
fund 


3 Former Vice President Biden’s platform on 
education beyond high school includes a 
recommendation to create such a grant program., 
https://joebiden.com/beyondhs/ 


4 Strong Evidence: Programs shown in well- 
conducted randomized controlled trials (RCTs), 
carried out in typical community settings, to produce 
sizable, sustained effects on important outcomes. 
This category requires replication—specifically, the 
demonstration of such effects in two or more RCTs 
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conducted in different implementation sites, or, 
alternatively, in one large multi-site RCT. 


5 Preliminary Evidence: The model has evidence 
based on a reasonable hypothesis and supported by 
credible research findings. Such evidence suggests 
the program may be an especially strong candidate 
for further research but does not yet provide 
confidence that the program would produce 
important effects if implemented in new settings. 


6 Promising Evidence: Interventions that have been 
evaluated in RCTs or rigorous quasi-experimental 
studies, and found to have positive effects that are 
sizable, but not yet conclusive (e.g., due to only 
short-term follow-up, a single-site study design, well- 
matched comparison groups but not randomization, 
or effects that fall short of statistical significance). 
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